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ABSTRACT 

The objectives of this program evaluation were to 
gather information and normative data about the students referred for 
psychoeducat ional evaluation; to determine the diagnostic profile of 
students referred for special education; and to use the information 
gathered to determine future program direction and possible program 
modifications. Students (N=55) in non-public schools referred for 
psychoeducat ional evaluation because of learning problems were 
evaluated. Students were evaluated using the Wechsler Intelligence 
Scale for Children III and the Wide Range Achievement Test-Revised 
(WRAT-R) . After the evaluation these students were assigned to one of 
three dispositions which determined an intervention of increasing 
intensity (n behalf of the student. The first level was 
teacher/psychologist consultation. The second level was 
multidiscipl inary teem meeting including the parent. The third level 
was a multidisciplinary team meeting with referral for an 
individualized educational planning meeting. Results indicated that 
there were no significant differences in psychologists* case 
dispositions using intelligence scores as dependent variables. As 
achievement scores in reading and spelling on the academic screening 
measure, the WRAT-R, yielded the only significant differences among 
case dispositions, a classroom based assessment of academic skills 
would be a viable alternative to the varied asses^aient batteries 
currently being administered. (ABL) 
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Abstract 

The focus of this study was fifty-five students in non-public 
schools referred for psychoeducational evaluation because of 
learning problems. After an evaluation these students were 
assigned one of three dispositions which determined an 
intervention of increasing intensity in behalf of the student • 
The first level was teacher/psychologist consultation. The second 
level was multidisciplinary team meeting including the parent and 
the third level was a multidisciplinary team meeting with referral 
for an individualized educational planning meeting. 

A MANOVA using the disposition of the student as an 
independent variable and WISC-III and WRAT-R scores as dependent 
variables yielded a significant difference between these referred 
for special education placement and those receiving consultation, 
but only in their reading and spelling scores on the WRAT-R. 

This most significant finding was one of several which will 
steer modifications in the role of our staff psychologists and the 
type of evaluations they perform. 
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This study was a program evaluation performed by 
two school psycholoc, ;sts who serve as supervisors of a 
school counseling and school psychology program for 
non-public schools in Phi ':.^dalphia. 

There were three main objectives for this program 
evaluation study. The first was to gather information 
and normative data about the population of those 
students referred for psychoeducational evaluation, the 
second was to determine the diagnostic profile of 
students re\erred for special education, and the third 
objective was to use the information gathered to 
determine future program direction and possible program 
modifications. 

With the advent of the new Pennsylvania special 
education regulations which have emphasized 
instructional support teams and regular education 
initiatives, there was a commitment on the part of the 
schools this agency serviced to maintain as many 
students as possible in regular classes. This would be 
accomplished by using the school psychologist as a 
consultant to the learning process with students 
displaying academic and/or behavioral difficulties. 
This commitment was in line with The National 
Association of School Psychologists' (NASP) historical 
concern regarding the required labeling of children for 
special services and the resultant value of those 
programs (Cobb 1990). 
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In addition, NASP has continually supported the 
objective of developing viable alternatives to the 
test-label-place model . 

In the face of continual seirvice cutbacks, these 
supervisors sought program analysis as an aiu co 
determine how to make the best use of the 
psychologists ' time . 

Performing school-based program evaluations must 
take into account the impossibility of random 
assignment, reduced options for the use of control 
groups, and the need for ingenuity in conducting 
statistical analysis (Sandoval, 1978) . Compromise in 
research design was faced by these authors in two prior 
school-based evaluations (Lavoritano & Segal 1992a, 
1992b) and like Sandoval these authors concluded that 
despite design limitations, it was possible to obtain 
usable data which contributed to program direction and 
enhanced decision making. 

Prior to embarking upon this program evaluation, 
several different evaluation methodologies were 
considered: 1) An evaluation to see that professional 
standards were maintained, i.e. proper permission forms 
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were obtained, feedbacks scheduled, classroom 
observation occurred etc* (Jackson & Pryzwansky, 1987) ; 
2' A qualitative evaluation using the school 
psychologist as a participant/observer (Davis, 1989) ; 
3} A systems/team approach that involved classification 
of the problem, planning, implementation, analysis and 
assessment (Fairchild, 1986; Maher 1978, 1979, Maher & 
Kratochwill 1980) ; 4) questionnaires sent to school 
principals (Senft & Smider, 1980) ; 5) A functional 
analysis of psychologist activities (Tomlinson, 1974) • 

After review of t: e varied program evaluation 
methodologies, the authors decided to combine an 
evaluation of psychological services with the gathering 
of diagnostic data. This information would be used to 
evaluate our staff psychologists* decision making to 
determine whether the time and money spent in formal 
psychoeducational evaluations was cost and program 
effective. In light of the research from The Learning 
Disabilitias Research Institute at the University of 
Minnesota (Ysseldyke et. al, 1983) which questioned the 
validity of traditional evaluation as a discriminator 
of learning disabled children from low achievers, 
scrutiny of our school psychological services was 
particularly warranted. 
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Method 
Subjects 

This study was run from January through June 
1992. During that time 128 students spanning grades 
one through eight received psychoeducational 
evaluations. All students attended non-public schools 
in the City of Philadelphia* 

The following information was gathered about the 
entire evaluation population: 

1) 68% were White, 21% Black, and 11% were Hispanic, 

2) 45% were from Chapter 1 schools, 3) 41% had been 
retained, 69% were receiving remedial reading and 39% 
were receiving math remediation, 4) 82% were referred 
for learning problems only, 7% were referred for 
learning/behavior problems, and 11% were referred for 
emotional/behavior problems; 5) 30% of the students 
with learning problems were referred to an lEP team for 
special education supports. Another 5% with 
learning/behavioral problems were referred to an lEP 
team for special education supports. 6) The majority of 
the students i.e. 65%, received consultative services. 

Students referred for gifted testing were not 
included in this study. 

The focus of this study was a sample of fifty-five 
(55) students referred for learning problems who had 
complete WISC — III and WRAT-R data. 
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The demographic breakdown of the sample was almost 
identical to the entire evaluation population: 1) 70% 
were white, 20% black and 10% Hispanic, 2) 51% were 
from Chapter I schools, 3) 45% had been retained, 72% 
were receiving remedial reading reading and 42% were 
receiving math remediation. 

Nine Pennsylvania state certified school 
psychologists participated in this program evaluation. 
Two were doctoral level psychologists, while the 
remaining seven were master's level psychologists. 
The psychologists' level of experience ranged from 
three to fifteen years of experience. 

Psychologists were asked to collect demographic 
data (gender, race, family status) , as well as school 
history information such as retention status, remedial 
services received (math, reading, speech/ language) on 
each child they evaluated. Prior to evaluation, 
students were first classified by the psychologist from 
information gathered by referral process as either 
having had a learning problem, a learning/behavior 
problem, or an emotional problem. After the 
evaluation, trie psychologists determined a disposition 
for each student they evaluated. There were three 
levels of intervention of varying intensity that the 
psychologist could utilize. The first level was to 
address the problem by teacher consultation. The next 
level was to coordinate efforts with a 

ERLC ^ 
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multi-disciplinary staff and parent involvement. The 
final step was a multi-disciplinary team meeting which 
would result in a referral for an individualized 
educational plan. 

A data collection form was created and distributed 
to all psychologists to facilitate the collection of 
information. 

After receiving signed parental consent forms as 
well as referral forms completed by the student's 
teacher, the psychologists were instructed to give 
their psychoeducational evaluations as usual and to 
record all data that was in the form of standard 
scores. 

Since the psychoeducational battery often included 
tests that used other quantifying measures such as 
instructional levels, grade or age equivalents without 
standard scores, this information was not included in 
the study and was a limitation of the research design • 

Generally, for the intellectual measure the 
psychologists utilized the WISC — III. Other 
intellectual assessments that were used conjunctively 
with the WISC~III included the PPVT-R, and the SIT-R. 
For achievement, a range of standardized tests were 
administered such as the Woodcock-Johnson 
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Psychoeducational Battery-Revised, the Woodcock Reading 
Mastery Test-Revised, and the Wide Range Achievement 
Test-Revised. The WRAT-R was the only test where 
standard scores ware consistently reported. On other 
measures, sometimes grade equivalents, percentiles, 
etc. were reported. Furthermore, not all tests were 
given to each child and selected achievement subtests 
were given to children as warranted. 

In the area of perceptual -motor skills, VMI scores 
were reported. 
Results 

Fifty-five students had complete WISC-III and 
WRAT-R data. The disposition of the student was used as 
the independent variable while all WISC — III and WRAT-R 
scores served as the dependent variables. The 
multivariate analysis of variance (MANOVA) yielded 
significant results (F==2.20, df 20/86, p<.01) by the 
Wilk's Lambda criterion for the three levels of the 
independent variable across the ten dependent 
variables. 

The follow-up procedure chosen to compare each of 
the case dispositions for ^ach dependent variable was 
the simultaneous confidence interval process* 
Ninety-five per cent simultaneous confidence intervals 
were constructed between case dispositions to determine 
which dependent variables contributed significantly to 
case dispositions. 

ERIC . 
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The results of these multiple comparisons are 
summarized in Table 1. In that table, the ten 
dependent variables are listed along with the three 
case dispositions. A line under two or more of the 
dispositions for a given variable indicates the case 
dispositions do not differ at the .05 level with 
respect to that variable. 



INSERT TABLE 1 HERE 



This comparison shows that the WRAT-R Reading 
(decoding) and Spelling (encoding) subtests were the 
only dependent diagnostic variables that demonstrated a 
significant difference between the groups, and only did 
that difference surface between the cases that were 
referred for placement versus the two groups that 
received the consultation dispositions. The WISC-— III 
profile of the students was relatively flat and not 
characteristic of the learning disabled students used 
in the validity study of the WISC — III (Wechsler, 
1991) . The means and the standard deviations of the 
dependent variables versus the three case dispositions 
are shown in Table 2. 
INSERT TABLE 2 HERE 
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Discussion 

In this program evaluation study, there was no 
significant difference in psychologists' case 
dispositions using intelligence scores as dependent 
variables. This finding was in line with Ysseldyke's 
research at the Learning Disabilities Research 
Institute where psychometric evaluations did not yield 
reliable differences between learning disabled children 
and low achieving students* 

As achievement scores in reading and spelling on 
the academic screening measure, the WRAT-R, yielded the 
only significant differences between case dispositions, 
a classroom based assessment of academic skills would 
be a viable alternative to the varied assessment 
batteries currently administered* Simple classroom 
measures of academic skills have shown to be reliable 
(Marston & Deno, 1981) and to yield high correlations 
with commercial tests in reading (Deno, Mirkin, & 
Chiang, 1982), spelling (Deno et. al. , 1982) and 
written expression (Deno, Marston, and Mirkin, 1982). 
With this in mind, these supervisors will recommend to 
the psychological staff to move toward the 
administration of curriculum based assessment 
methodologies. 
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This assessment procedure would be less labor 
intensive, more valid and reliable, and would free the 
psychologist to do more consultation on effective 
learning strategies with parents and teachers. 

Too often the formal assessment process focuses on 
validating what the teacher already knows about the 
student. Since two-thirds of the students in this 
study who were formally evaluated remained in the 
classroom, early and continuous indirect support to the 
student via intervention teams including the child's 
teacher(s), parent (s) , and the psychologist will be 
another recommendation to the psychological staff. 

Presently, the formal comprehensive 
psychoeducational evaluations have a place in the 
assessment process, as they are needed for the lEP team 
to consider educational supports, but the consultative 
role of the school psychologist will become the more 
important role for our psychological staff. As 
supervisors, we must continue to secure training that 
will hone the consultative skills for our staff. 
NASP has been strongly encouraging school psychologists 
to move away from the test-label -place role and to move 
toward a more consultative model for a number of years. 

Curriculum-based and portfolio assessment as well 
as the idea of intervention teams are not new ideas. 
However, it is the formalized process of program 
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evaluation that strengthened the coitunitinent of these 
supervisors and their staff to make the necessary 
changes in the delivery of school psychological 
services, as old habits die hard. 
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Means and Standard deviations of the dependent variables 
versus the case disposition 

Case Disposition 



PC 
N=21 



MDC 
N=16 



MD/IEP 
N=18 



ALL GROUPS 
N=55 



Variable 
WISC-III VIQ 
Mean 

Standard Deviation 



88.95 
13.95 



92.94 
12 .79 



89.11 
10. 23 



90. 16 
12.40 



WISC-III PIQ 
Mean 

Standard Deviation 



87.57 
8.95 



90.94 
6. 59 



85. 55 
17 .78 



87.89 
12.08 



WISC-III FSIOQ 
Mean 

Standard Deviation 



87 . 19 
11.41 



91.31 
9.02 



86. 16 
11.96 



88.05 
10.98 



WISC-III VCI 
Mean 

Standard Deviation 



89.52 
13.44 



94.94 
13 . 02 



89. 11 
10.85 



90.96 
12.56 



WISC-III POI 
Mean 

Standard Deviation 



86.76 
10.58 



90. 50 
5. 64 



85. 16 
17.41 



87.33 
12.26 



WISC-III FDI 
Mean 

Standard Deviation 



91. 57 
14.62 



85.94 
11.73 



91.72 
9.10 



89.98 
12.26 



WISC-III PSI 
Mean 

Standard Deviation 



98. 19 
13.83 



101.81 
12.23 



94.39 
14.50 



98. 00 
..3.68 



WRAT-R Reading 
Mean 

Standard Deviation 



84.57 
12 .99 



83.12 
17.55 



68 .78* 
12 .76 



78.98 
15.84 



WRAT-R Spelling 
Mean 

Standard Deviation 



84.48 
10.57 



86.75 
15. 13 



73.78** 
14.10 



81.64 
14. 11 



WRAT-R Arithmetic 
Mean 

Standard Deviation 



91. 19 
16.38 



92.56 
13.05 



86. 06 
11.80 



89.89 
14.08 



*p<.01, **p=.01 
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Suitimary of the multiple comparisons 
disposition main effect 


for 


the case 




Dependent Variable 




Program 


Wise III VIQ 


PC 


MDC 


MD/IEP 


Wise III PIQ 


PC 


MDC 


MD/IEP 


Wise III FSIQ 


PC 


MDC 


MD/IEP 


wise III vei 


PC 


MDC 


MD/IEP 


wise III PCI 


PC 


MDC 


MD/IEP 


wise III FDI 


PC 


MDC 


MD/IEP 


wise III PSI 


PC 


MDC 


MD/IEP 


WRAT-R Reading 


PC 


MDC 


MD/IEP 


WRAT-R Spelling 


PC 


MDC 


MD/IEP 


WRAT-R Arithmetic 


PC 


MDC 


MD/IEP 



PC = case managed by Psychologists' consultation 

MDC = case managed by Multidisciplinary consultation 

MD/IEP = case managed by Multidisciplinary team meeting with 
referral to the lEP team for educational supports. 
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